IO_Demo benchmarking configuration by EdHone · Pull Request #232 · MetOffice/lfric_core

EdHone · 2026-01-21T09:42:04Z

PR Summary

Sci/Tech Reviewer: @andrewcoughtrie
Code Reviewer: @mo-lucy-gordon

This change introduces a flexible benchmarking configuration for the IO_Demo app. The new functionality enables a user to specify a number of fields to write, along with a number of seconds to wait during a step to mimic simulation time. This simple configuration should enable us to exercise the LFRic I/O capability for a number of different use-cases, including writing data at scale.
The changes include:

source changes in io_demo to enable new benchmark capability
changes to build the app which enable the use of the sleep() intrinsic for all compilers
changes to io_demo metadata which allow the app to be controlled a bit more intuitively
tweaks to the io_demo algorithm layer to enable it to be stable at scale
tweaks to launch-exe to enable running with XIOS servers in the lfric_core rose-stem
new rose-stem tests, along with a new io_demo_benchmark group which runs at large-ish scale (C224)

closes IO_Demo benchmark configuration #216

Code Quality Checklist

(Some checks are automatically carried out via the CI pipeline)

I have performed a self-review of my own code
My code follows the project's
style guidelines
Comments have been included that aid understanding and enhance the
readability of the code
My changes generate no new warnings

Testing

I have tested this change locally, using the LFRic Core rose-stem suite
If required (e.g. API changes) I have also run the LFRic Apps test suite
using this branch
If any tests fail (rose-stem or CI) the reason is understood and
acceptable (e.g. kgo changes)
I have added tests to cover new functionality as appropriate (e.g. system
tests, unit tests, etc.)
Any new tests have been assigned an appropriate amount of compute resource
and have been allocated to an appropriate testing group (i.e. the
developer tests are for jobs which use a small amount of compute resource
and complete in a matter of minutes)

trac.log

Test Suite Results - lfric_core - lfric_core-216-iodemo-bench-vernier/run7

Suite Information

Item	Value
Suite Name	lfric_core-216-iodemo-bench-vernier/run7
Suite User	edward.hone
Workflow Start	2026-01-21T10:28:40
Groups Run	developer

Dependency	Reference	Main Like
lfric_core	EdHone/lfric_core@216-iodemo-bench-vernier	False
SimSys_Scripts	MetOffice/SimSys_Scripts@2025.12.1	True

Task Information

✅ succeeded tasks - 377

Test Suite Results - lfric_core - lfric_core-216-iodemo-bench-vernier/run6

Suite Information

Item	Value
Suite Name	lfric_core-216-iodemo-bench-vernier/run6
Suite User	edward.hone
Workflow Start	2026-01-21T10:17:45
Groups Run	io_demo_benchmark

Dependency	Reference	Main Like
lfric_core	EdHone/lfric_core@216-iodemo-bench-vernier	False
SimSys_Scripts	MetOffice/SimSys_Scripts@2025.12.1	True

Task Information

✅ succeeded tasks - 10

Security Considerations

I have reviewed my changes for potential security issues
Sensitive data is properly handled (if applicable)
Authentication and authorisation are properly implemented (if applicable)

Performance Impact

Performance of the code has been considered and, if applicable, suitable
performance measurements have been conducted

AI Assistance and Attribution

Some of the content of this change has been produced with the assistance
of Generative AI tool name (e.g., Met Office Github Copilot Enterprise,
Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the
Simulation Systems AI policy
(including attribution labels)

Documentation

Where appropriate I have updated documentation related to this change and
confirmed that it builds correctly

PSyclone Approval

If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel
interface, optimisation scripts, LFRic data structure code) then please
contact the
tooscollabdevteam@metoffice.gov.uk

Sci/Tech Review

I understand this area of code and the changes being added
The proposed changes correspond to the pull request description
Documentation is sufficient (do documentation papers need updating)
Sufficient testing has been completed

Please alert the code reviewer via a tag when you have approved the SR

Code Review

All dependencies have been resolved
Related Issues have been properly linked and addressed
CLA compliance has been confirmed
Code quality standards have been met
Tests are adequate and have passed
Documentation is complete and accurate
Security considerations have been addressed
Performance impact is acceptable

MatthewHambley

Presumed code owner review: The "allow everything" argument is a bit over broad but for a demo application and until we can work out how we actually want to handle the issue of "sleep", I am content.

andrewcoughtrie

This all looks fine to me. As mentioned in person could you open an issue to create a sleep function that wraps around C usleep or similar.

EdHone · 2026-01-23T15:21:05Z

This all looks fine to me. As mentioned in person could you open an issue to create a sleep function that wraps around C usleep or similar.

Done - #241

EdHone · 2026-02-02T10:50:00Z

I've just added 2 more commits which just change some of the rose-stem infrastructure around the tests:

to copy the vernier output to the task output dir
to only output a single file from vernier

I don't think these changes should require further science review, so I'll pass over to @mo-lucy-gordon for CR

james-bruten-mo

Weird indenting, but otherwise rose-stem is all fine

james-bruten-mo · 2026-02-04T11:40:27Z

rose-stem/templates/runtime/generate_runtime_application.cylc

    'test -f $CYLC_TASK_WORK_DIR/timer.txt && '~
-        'cp $CYLC_TASK_WORK_DIR/timer.txt $TASK_OUTPUT_DIR || true'
+        'cp $CYLC_TASK_WORK_DIR/timer.txt $TASK_OUTPUT_DIR || true',
+    'test -f $CYLC_TASK_WORK_DIR/vernier-output* && '~


Some weird indenting

It looks better in the full file, promise! Also follows the pattern from the other parts of the file

applications/io_demo/rose-meta/lfric-io_demo/HEAD/rose-meta.conf

applications/io_demo/source/driver/io_benchmark/io_benchmark_setup_mod.f90

mo-rickywong · 2026-02-04T13:27:29Z

applications/io_demo/source/driver/io_benchmark/io_benchmark_setup_mod.f90

+    type(namelist_type),         pointer :: io_nml
+
+    io_nml => modeldb%configuration%get_namelist('io')
+    call io_nml%get_value('diagnostic_frequency', diagnostic_frequency)


Suggested change

type(namelist_type), pointer :: io_nml

io_nml => modeldb%configuration%get_namelist('io')

call io_nml%get_value('diagnostic_frequency', diagnostic_frequency)

diagnostic_frequency = modeldb%config%io%diagnostic_frequency()

applications/io_demo/source/driver/io_benchmark/io_benchmark_setup_mod.f90

applications/io_demo/source/driver/io_benchmark/io_benchmark_step_mod.x90

mo-rickywong · 2026-02-04T13:38:19Z

New config_type is on main, so bringing up to head of main will show these changes. Please access the new namelist variables in a similar manner.

EdHone · 2026-02-04T19:29:42Z

I've addressed all your comments @mo-rickywong, and rebased the branch up the head of main

mo-rickywong · 2026-02-05T08:52:17Z

One other, thing, does this need an upgrade-macro? @james-bruten-mo

mo-rickywong · 2026-02-05T09:05:41Z

applications/io_demo/source/driver/io_demo_driver_mod.f90

    !=======================================================================
-    if (multifile_io) then
+    files_init_ptr => null()
+    if(multifile_io) then


Suggested change

if(multifile_io) then

if (multifile_io) then

james-bruten-mo · 2026-02-05T09:09:39Z

One other, thing, does this need an upgrade-macro? @james-bruten-mo

Thanks Ricky - I think I've discussed this one with Ed before. Technically it should have an upgrade macro, but as it's only in io_demo, then it's probably ok - I imagine no one is making a standalone io_demo suite!

stevemullerworth

Code owner review: all fine

EdHone requested review from MatthewHambley, andrewcoughtrie and stevemullerworth as code owners January 21, 2026 09:42

github-actions bot assigned EdHone Jan 21, 2026

EdHone marked this pull request as draft January 21, 2026 09:49

EdHone marked this pull request as ready for review January 21, 2026 10:42

MatthewHambley approved these changes Jan 21, 2026

View reviewed changes

andrewcoughtrie approved these changes Jan 23, 2026

View reviewed changes

github-actions bot requested a review from mo-lucy-gordon January 23, 2026 15:02

EdHone requested review from james-bruten-mo and mo-rickywong as code owners February 2, 2026 10:35

james-bruten-mo approved these changes Feb 4, 2026

View reviewed changes

mo-rickywong requested changes Feb 4, 2026

View reviewed changes

EdHone added 14 commits February 4, 2026 14:51

Add source code changes from previous Trac branch

51ad658

Remove old timer implementation

e51a8cc

Bring in metadata from previous branch

3569a1a

Use 'native' Fortran sleep

dae42a7

Fix upgrade macro

3a97bd1

Macro whitespace

d1b0048

Macro whitespace

30f3842

Manual implementation of metadata - test suite runs but tests broken

5c9f123

Fix metadata issues

8149534

Fix builds and suite running

073c302

small benchmark cases in developer suite

90d94df

Interim test configs working

250ce13

Working tests for benchmark case

c58ff99

Clean up branch

e078f19

EdHone added 4 commits February 4, 2026 14:54

Fixes for developer suite

ec5a07b

Final implementation

6747bc8

change vernieer output mode

07f3b88

RW comments - new namelist API and rose metadata triggers

2974558

EdHone force-pushed the 216-iodemo-bench-vernier branch from 6205f6e to 2974558 Compare February 4, 2026 15:09

More efficient function space assignment

f489e4c

EdHone requested a review from mo-rickywong February 4, 2026 19:29

fix indentation

cc89e8a

mo-rickywong reviewed Feb 5, 2026

View reviewed changes

stevemullerworth reviewed Feb 5, 2026

View reviewed changes

Conversation

EdHone commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Code Quality Checklist

Testing

trac.log

Test Suite Results - lfric_core - lfric_core-216-iodemo-bench-vernier/run7

Suite Information

Task Information

Test Suite Results - lfric_core - lfric_core-216-iodemo-bench-vernier/run6

Suite Information

Task Information

Security Considerations

Performance Impact

AI Assistance and Attribution

Documentation

PSyclone Approval

Sci/Tech Review

Code Review

Uh oh!

MatthewHambley left a comment

Choose a reason for hiding this comment

Uh oh!

andrewcoughtrie left a comment

Choose a reason for hiding this comment

Uh oh!

EdHone commented Jan 23, 2026

Uh oh!

EdHone commented Feb 2, 2026

Uh oh!

james-bruten-mo left a comment

Choose a reason for hiding this comment

Uh oh!

james-bruten-mo Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

EdHone Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mo-rickywong Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mo-rickywong commented Feb 4, 2026

Uh oh!

EdHone commented Feb 4, 2026

Uh oh!

mo-rickywong commented Feb 5, 2026

Uh oh!

mo-rickywong Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

james-bruten-mo commented Feb 5, 2026

Uh oh!

stevemullerworth left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

EdHone commented Jan 21, 2026 •

edited

Loading